Neural networks are a form of architecture which enables a computer to closely approximate human thought processes. One form of neural network architecture enables single instruction stream, multiple data stream (SIMD) operations which allow a single command to direct a number of processors, and hence data sets, simultaneously.
There are several important practical problems that cannot be solved using existing, conventional algorithms executed by traditional, conventional computers. These problems are often incompletely specified and are characterized by many weak constraints requiring large search spaces.
The processing of primary cognitive information by computers, such as computer speech recognition, computer vision, and robotic control, fall into this category. Traditional computational models bog down to the point of failure under the computational load if they are tasked to solve these types of problems. Yet animals perform these tasks using neurons that are millions of times slower than transistors. A neuron performs a weighted sum of inputs, which may be described as .SIGMA.W.sub.ij O.sub.i, where W.sub.ij is a value drawn from "memory" and o.sub.i is an input value. A SIMD multiply/accumulate function performs this operation.
Feldman's 100-step rule, which is an argument for massively parallel computing, states that a "human" cognitive process having a time of 500 msec. can be accomplished in 5 msec neuron switching time. If the "switching" time is slow, a fast system may nevertheless be constructed with a large number of "switches". This implies that there are two vastly different computational models at work. It also suggests that in order to build computers that will do what nervous systems do, the computers should be structured to emulate animal nervous systems. A SIMD system is a computing system which is designed to emulate a massively parallel neural network.
A nervous system, and a neurocomputational computer, is characterized by continuous, non-symbolic, and massively parallel structure that is fault-tolerant of input noise and hardware failure. Representations, i.e., the input, is distributed among groups of computing elements, which independently reach a result or conclusion, and which then generalize and interpolate information to reach a final output conclusion.
Put another way, connectionist/neural networks search for "good" solutions using massively parallel computations of many small computing elements. The model is one of parallel hypothesis generation and relaxation to the dominant, or "most-likely" hypothesis. The search speed is more or less independent of the size of the search space. Learning is a process of incrementally changing the connection (synaptic) strengths, as opposed to allocating data structures. "Programming" in such a neural network is by example.
In the case where total, or near total, connectivity (30% or higher) of communication node units is constructed, a great deal of time is spent waiting for the individual processors to communicate with one another after the data set has been acted upon. Known processing units do not provide a mechanism for allowing computation to occur on a subsequent data set until all processor nodes have transmitted their previously acted upon data.